Using the plantcomNGPN R package

1. Setup

Overview

The plantcomNGPN package was designed to help compile, query and summarize plant community monitoring data collected by the Northern Great Plains Inventory and Monitoring Network (NGPN) data and stored in the FFI (FEAT/FIREMON Integrated) database.

There are a couple of ways to import NGPN FFI data into R, and which will be demonstrated in code below:

  1. Import FFI tables from a local database installation in SQL Server Management Studio (SSMS).
    This approach requires having SQL Server Express installed, which is not available in Software Center and requires elevated privileges from IT to install. SQL Server Express may prohibited in the near future. This approach involves restoring the .bak files, which are exports of the FFI database for each park from the SQL Server database, as a local database copy on a given computer. See SSMS > Local SSMS tab for setup instructions. Once that’s completed, continue to R package setup.
  2. Import FFI tables from SQL Server.
    Ideally this will be the main way that NGPN staff import FFI data into R, as it will be importing live data, allowing for real-time QAQC, fixing any errors found, then rerunning the import. Likely each user will need permissions set on the server with their Active Directory account (so NPS or NPS-partners only). SSMS will need to be installed, which is available in Software Center, but SQL Server Express isn’t needed. We will continue to request this access through MB and/or FFI developers until we have it. See SSMS > Server SSMS tab for setup instructions. Once that’s completed, continue to R package setup.
  3. Import csvs of FFI database tables.
    The importData() function in this package can export a zip file containing csvs of all of the raw FFI database tables. Once that zip file is created, it can be used to import FFI data without the need of SSMS or SQL Server Express. This option allows non-NPS users to work with NPGN data. If this is the option you’re using, go to R package setup tab for details on R package installation and setup.
  4. Import csvs of the FFI views.
    In addition to importing and exporting FFI tables, the importData() function compiles the FFI raw tables into flattened stand-alone views of each FFI protocol (e.g. Cover Points, Density Belts, Trees, etc.), which are used by package functions to further query, summarize, and visualize the data. The views can be exported as a zip file in the importData() function. The zip file of views can be imported via importViews() function. This option is the fastest way to import NPGN data into R, and allows non-NPS users to more easily work with NPGN data. These views are intended to be analysis-ready. Any feedback on how to improve their usability is welcome. If this is the option you’re using, go to R package setup tab for details on R package installation and setup.

Currently, only import and getter functions exist in the R package. In time, more features will be added, such as functions that perform common summaries and visualize data, and this tutorial will be updated accordingly. Feedback is always welcome on how to improve this tutorial and the R package in general.

Additionally, an automated QC report that checks NPGN FFI data for potential missing data and errors is being developed separately.

SSMS
Local SSMS

Once this step is complete, users can import FFI database tables into R using the importData() function and the name of the database in SSMS (see ImportData tab).

Step 1. Install SQL Server Management Studio
Instructions for installing SQL Server Management Studio (SSMS) are currently being developed. Once those are ready, they will be linked here. The best directions currently (though a few years out of date) can be found here: SOP_SQLServer_Setup.docx, and are only available to DOI users.
Step 2. Restore database from .bak file in SSMS

To restore a database from a .bak file in SSMS, you can either restore through the file menu following the screencast or run the SQL code in SSMS below.

File Menu option:

SQL Code option: The SQL code below will restore a .bak file to a database named “FFI_RA_AGFO”. To use the code below, you only need to change “FFI_RA_AGFO” to match the name of your database (i.e., whatever precedes the .bak in the file name), and change the C:\temp\path to the path where your .bak file lives. Note that files on OneDrive often cause issues. It’s best to restore from locations directly on your C:\ drive or a server.

-- Variables to declare and modify for different database name and file locations
DECLARE @DBNAME NVARCHAR(MAX) = 'FFI_RA_AGFO'
DECLARE @FilePathOrig NVARCHAR(MAX) = 'C:\\temp\\'
DECLARE @FilePathNew NVARCHAR(MAX) = 'C:\\Program Files\\Microsoft SQL Server\\MSSQL15.SQLEXPRESS\\MSSQL\\DATA\\';
DECLARE @SQL NVARCHAR(MAX);

-- Run remaining query to restore database
--USE [master]

DECLARE @DATA NVARCHAR(MAX) = @FilePathNew + @DBNAME + '.mdf';
DECLARE @LOG NVARCHAR(MAX) = @FilePathNew + @DBNAME + '.ldf';
DECLARE @DBFULLPATH NVARCHAR(MAX) = @FilePathOrig + @DBNAME + '.bak';
DECLARE @DBLOG NVARCHAR(MAX) = @DBNAME + '_log';

USE[master]

SET @SQL =
  ' RESTORE DATABASE ['+@DBNAME+'] FROM DISK = '''+@DBFULLPATH+''' WITH FILE = 1, MOVE '''+@DBNAME+''' TO '''+@DATA+''',
    MOVE '''+@DBLOG+''' TO '''+@LOG+''',
    NOUNLOAD, STATS = 5'
EXEC(@SQL)
GO


Server SSMS

…hopefully

R package setup

If you’ve never installed an R package from GitHub, you’ll need to complete the following steps to install the plantcomNGPN R package. Once completed, move on to the ImportData and other function tabs for instructions on using the R package.

Step 1. Install R, RStudio, and RTools44 in Software Center

RTools is needed to install R packages from GitHub via devtools, and it only works with R versions 4.4.x. While R 4.5 is available on Software Center, the matching RTools45 isn’t available yet. Until that changes, link RStudio to the latest version of R 4.4 (I’m currently using R 4.4.3).

Step 2. Set RTools44 path
Unfortunately Software Center installs RTools44 in C:/Program Files/, not C:/, which is where RStudio looks for it by default. The following code helps RStudio find RTools. You may occasionally have to rerun this code (except for the usethis line), so keep it handy. You know when you have to rerun the code when you try to rebuild a package, and a window pops up to ask if you want to install missing build files.

  • Install usethis and open .Renviron: If you don’t have the usethis package, install it first (first line below). Once installed, open the .Renviron using the second line below.

    install.packages('usethis')
usethis::edit_r_environ()

  • Set RTools path: Next you’re going to tell .Renviron where to find RTools upon RStudio opening by pasting the code below into that file, and saving it. Now close/reopen RStudio, and you should be all set.

    Sys.setenv(PATH = paste("C:\\PROGRA~1\\Rtools44\\bin", Sys.getenv("PATH"), sep=";"))
Sys.setenv(BINPREF = "C:\\PROGRA~1\\Rtools44\\mingw_$(WIN)\\bin\\")

Step 3. Install devtools package in R: 
install.packages('devtools')
Step 4. Set up GitHub account if a new user

To install R packages using devtools, you need to have a GitHub user account and active token. Instructions for setting up a GitHub account and connecting it to RStudio are on the IMD Advanced R training website from 2022. Go to Day 4: Version Control > Git and RStudio and complete the steps described. There’s a lot of other good info on using GitHub in other tabs too.

Step 5. Install plantcomNGPN from GitHub

Note that whenever the plantcomNGPN package is updated, you can rerun this code to install the latest version.

library(devtools)
install_github("KateMMiller/plantcomNGPN")

If you don’t want to have a GitHub account, you can also install packages from GitHub by downloading a .zip file of the code (often called forking a repo), then running the code below.

install.packages("plantcomNGPN.zip")
  • Troubleshooting GitHub package installation

    If you’re unable to install the R package via GitHub (often an error about permission being denied, or cannot open URL, download the following script from my OneDrive and open it in R: fix_TLS_inspection.R. Download by left clicking on the link, then click in the download arrow in the top left of the screen. Save it to your machine, then open it in RStudio.

    Once this script is open in R Studio, press Control + A to select all of the code. Then Control + Enter to run all of the code. Assuming you don’t return any errors, you should be able to install from GitHub. Now try to reinstall plantcomNGPN. If you’re still running into issues, it could be that devtools is missing some package dependencies, which the error message will often mention. Install any missing packages and then try installing plantcomNGPN again. If you’re still not successful, send Kate Miller () a screenshot of the error and she’ll help you troubleshoot.

Step 6. Load plantcomNGPN R package 
library(plantcomNGPN)


2. R package Intro

Import functions
Overview
Once you have the R package installed, you can import and start working with the data. There are 2 import functions in the plantcomNGPN package:
  • importData(): imports raw FFI tables as individual data frames, either from SSMS or as a zip file of csvs. If multiple databases are specified in the import, the tables will be rowbinded across parks. This function also includes features to export data as a zip file of csvs, which can then be used in the following function.
  • importViews(): a streamlined function that imports a zip file of flattened views for each FFI protocol.
importData()
By default, the importData() function imports FFI tables and only returns the flattened views as data frames in an environment called VIEWS_NGPN. This minimizes memory usage and keeps a tidy R workspace. R package functions are then designed to work seamlessly with the views, regardless of whether they’re in the VIEWS_NGPN environment or global environment. See Working with the data tab for how to work with data.frames housed within the VIEWS_NGPN environment.

  • Import data for 1 database using local install in SSMS

    Note that R is not able to connect to files on Sharepoint or MS Teams (Teams also stores all files on Sharepoint). That means you need to store data package files on your local machine or on a server. The default option for importing data will add the data package views (i.e., flatfiles) to an environment called FFI_tables to your Environment work space (i.e. Environment tab in top right panel).

    library(plantcomNGPN)
importData(dbname = "FFI_RA_AGFO")

  • Import data for multiple databases using local installs in SSMS

    The code below will bind like data/tables across parks together to have 1 large view

    importData(type = 'local',
           dbname = c("FFI_RA_AGFO", "FFI_RA_BADL", "FFI_RA_DETO", "FFI_RA_FOLA",
                      "FFI_RA_FOUS", "FFI_RA_JECA", "FFI_RA_KNRI", "FFI_RA_MNRR",
                      "FFI_RA_MORU", "FFI_RA_SCBL", "FFI_RA_THRO", "FFI_RA_WICA"))

  • Import FFI tables as zip

    The code below will import a zip file of THRO data exported by the importData() function (see options below). This approach does not require SSMS to be installed.

    importData(type = 'csv', import_path = "C:/temp/FFI_RA_THRO.zip")
importViews()

  • Import FFI views as a zip

    The importViews() function is the fastest and least memory hungry way to import FFI data. Someone will have to have generated the views by first importing raw FFI tables, and then exporting the views. Once that’s completed, this import function is recommended.

    importViews(import_path = "C:/temp/NGPN_FFI_views_20250710.zip")
importData() Options
Export tables or views to zip files.

  • Export FFI tables to zip

    If you want to export the raw tables into a zip file of csvs, simply add export_tables = T. If no export_path is specified, the zip file will be saved in your working directory. The file name and path of the zip file will be printed in your console. Note that exporting can be slow, especially if you loaded multiple databases.

    importData(type = 'local', 
           dbname = c("FFI_RA_AGFO", "FFI_RA_BADL", "FFI_RA_DETO", "FFI_RA_FOLA",
                      "FFI_RA_FOUS", "FFI_RA_JECA", "FFI_RA_KNRI", "FFI_RA_MNRR",
                      "FFI_RA_MORU", "FFI_RA_SCBL", "FFI_RA_THRO", "FFI_RA_WICA"), 
           export_tables = T, export_path = "C:/temp/")

  • Export views to zip

    Even better, is exporting the views into a zip file of csvs. To do that, add export_views = T. If no export_path is specified, the zip file will be saved in your working directory. The file name and path of the zip file will be printed in your console. This zip file is then the fastest way to import FFI data into R using the importViews() function instead.

    importData(type = 'local', 
           dbname = c("FFI_RA_AGFO", "FFI_RA_BADL", "FFI_RA_DETO", "FFI_RA_FOLA",
                      "FFI_RA_FOUS", "FFI_RA_JECA", "FFI_RA_KNRI", "FFI_RA_MNRR",
                      "FFI_RA_MORU", "FFI_RA_SCBL", "FFI_RA_THRO", "FFI_RA_WICA"), 
           export_views = T, export_path = "C:/temp")

  • Return raw tables too

    There are times when it helps to view the raw FFI data tables, such as for troubleshooting an issue in the R package, or in figuring out where an error in the data was introduced and need to be fixed. To return the raw FFI tables in addition to the views, specify keep_tables = TRUE.

    importData(type = 'local', 
           dbname = c("FFI_RA_AGFO", "FFI_RA_BADL", "FFI_RA_DETO", "FFI_RA_FOLA",
                      "FFI_RA_FOUS", "FFI_RA_JECA", "FFI_RA_KNRI", "FFI_RA_MNRR",
                      "FFI_RA_MORU", "FFI_RA_SCBL", "FFI_RA_THRO", "FFI_RA_WICA"), 
           keep_tables = T)

  • Return views to global environment 

    importData(type = 'local', 
           dbname = c("FFI_RA_AGFO", "FFI_RA_BADL", "FFI_RA_DETO", "FFI_RA_FOLA",
                      "FFI_RA_FOUS", "FFI_RA_JECA", "FFI_RA_KNRI", "FFI_RA_MNRR",
                      "FFI_RA_MORU", "FFI_RA_SCBL", "FFI_RA_THRO", "FFI_RA_WICA"), 
           new_env = F)
Working with the Data

The functions in the plantcomNGPN package are designed to work with the views in VIEWS_NGPN, and are the best way to interact with the data to query by park, site, site type, year, parameter, etc. However, if you want to look at the views without using the package functions, you can use the following code.

# See list of the views
names(VIEWS_NGPN)
##  [1] "Disturbance_History"      
##  [2] "Cover_Points_metric"      
##  [3] "Trees_metric"             
##  [4] "SampleEvents"             
##  [5] "Density_Belts_metric"     
##  [6] "Surface_Fuels_Fine"       
##  [7] "Surface_Fuels_1000Hr"     
##  [8] "Density_Quadrats_metric"  
##  [9] "Cover_Species_Composition"
## [10] "Taxa_Table"               
## [11] "Surface_Fuels_Duff"       
## [12] "MacroPlots"
# Inspect the cover point view
View(VIEWS_NGPN$Cover_Points_metric)

#head(VIEWS_NGPN$Cover_Points_metric)

# Assign the coverpoint view to the object covpt, which now behaves like a data.frame.
covpt <- VIEWS_NGPN$Cover_Points_metric

Additionally, if you want to view the raw FFI data tables, and you imported the data into the NPGN_tables environment, you can access them with the code below:

importData(type = 'local', 
           dbname = c("FFI_RA_AGFO", "FFI_RA_BADL", "FFI_RA_DETO", "FFI_RA_FOLA",
                      "FFI_RA_FOUS", "FFI_RA_JECA", "FFI_RA_KNRI", "FFI_RA_MNRR",
                      "FFI_RA_MORU", "FFI_RA_SCBL", "FFI_RA_THRO", "FFI_RA_WICA"), 
           keep_tables = T)

# See list of FFI tables
sort(names(NGPN_tables))

# Inspect the MacroPlot table
View(NGPN_tables$MacroPlot)

# Assign the MacroPlot table to the object macro
macro <- NGPN_tables$MacroPlot
Getting help

Getting (and improving) help

The functions in plantcomNGPN have help documentation like any R package. To view the help, you can go to the Packages tab and click on plantcomNGPN (see below). That will show you all the functions in the package. Clicking on individual functions will take you to the help documentation for that function.

You can also see the help of a function by running, for example: 

?importData

If plantcomNGPN isn’t loaded yet, you’d run: 

?plantcomNGPN::importData

Each function’s help includes a Description, Usage (i.e. function arguments and their defaults), Argument options/definitions, and several examples showing how the function can be used.

This is where you come in! If you notice typos or can think of better descriptions, examples, error messages, etc., please send them my way! After we’re more comfortable with R packages and get versed on GitHub, you’ll be able to make those changes directly in the package. For now, you can just send me your suggestions and I’ll make the changes.

Finally, if you ever want to peak under the hood at the function, you can view it several ways.
  1. Keep F2 key pressed and click on the function name in R. This trick works for many but not all functions in R.

  2. View code in the GitHub katemmiller/plantcomNGPN repo. The functions are in the R folder.

3. getter Functions